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Background and purpose Patient-reported outcome measures 
(PROMs) are used by some arthroplasty registries to evalu- 
ate results after surgery, but non-response may bias the results. 
The aim was to identify a potential bias in the outcome scores 
of subgroups in a cohort of patients from the Danish Shoulder 
Arthroplasty Registry (DSR) and to characterize non-responders. 

Methods Patient-reported outcome of 787 patients operated in 
2008 was assessed 12 months postoperatively using the Western 
Ontario Osteoarthritis of the Shoulder (WOOS) index. In Janu- 
ary 2012, non-responders and incomplete responders were sent 
a postal reminder. Non-responders to the postal reminder were 
contacted by telephone. Total WOOS score and WOOS subscales 
were compared for initial responders (n = 509), responders to the 
postal reminder (n = 156), and responders after telephone contact 
(n = 27). The predefined variables age, sex, diagnosis, geographi- 
cal region, and reoperation rate were compared for responding 
and non-responding cohorts. 

Results A postal reminder increased the response rate from 
65% (6% incomplete) to 80% (3% incomplete) and telephone 
contact resulted in a further increase to 82% (2% incomplete). We 
did not find any statistically significant differences in total WOOS 
score or in any of the WOOS subscales between responders to 
the original questionnaire, responders to the postal reminder, and 
responders after telephone contact. However, a trend of worse 
outcome for non-responders was found. The response rate was 
lower in younger patients. 

Interpretation Non-responders did not appear to bias the 
overall results after shoulder replacement despite a trend of worse 
outcome for a subgroup of non-responders. As response rates rose 
markedly by the use of postal reminders, we recommend the use 
of reminders in arthroplasty registries using PROMs. 



Patient-reported outcome measures (PROMs) after orthope- 
dic surgery are used by some registries (Malchau et al. 2005, 
Goodfellow et al. 2010, Karrholm 2010, Knutson and Rob- 
ertsson 2010, Rolfson et al. 2011, Rasmussen et al. 2012). 
Uncertainty about outcome for patients who do not respond 
may lead to biased conclusions about the effects of interven- 
tions (Gluud 2006). 

2 clinical studies of non-responders to mail surveys after 
total knee arthroplasty found that they had poorer functional 
outcome and less satisfaction than responders (Kim et al. 
2004, Kwon et al. 2010). The same was found in a cohort of 
patients with rotator cuff tears (Norquist et al. 2000). Solberg 
et al. (201 1) traced non-responders to enquiries from a clinical 
spine surgery registry and found that there was similar out- 
come in responders and non-responders. 

The reliability of data in a registry is crucial for the use of 
data in research and for extrapolation of data to clinical set- 
tings. The main aim of this study was to identify a potential 
bias in outcome scores of subgroups in a cohort of patients 
from the Danish Shoulder Arthroplasty Registry (DSR). A 
second aim was to characterize non-responders. 



Patients and methods 

The DSR was established in 2004. It is a mandatory, national 
registry that collects information on all primary and revision 
arthroplasties of the shoulder joint. The surgeon reports data 
on the operation through an internet-based reporting system 
(Rasmussen et al. 2012). Patient-reported functional outcome 
is assessed by mail survey 12 months postoperatively using 
the Western Ontario Osteoarthritis of the Shoulder (WOOS) 
index. The WOOS is a disease-specific quality of life mea- 
surement tool based on 19 questions to be answered on a 
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Operations registered in DSR in 2008 
n=846 



encouraged to respond to the postal question- 



Excluded (n=59): 

- no CPR number, 4 

- no valid address in the CPR, 1 

- duplicate registrations, 2 

- error reportings, 1 

- dead in the first 12 months postop., 28 

- revision in the first 12 months postop., 23 



Included in the analysis 
n=787 



Respondents 
n=509 



Non-respondents 
n=278 



Complete 
questionnaires 
n=465 



Incomplete 
questionnaires | — 10- 
n=44 



Dead before 
January 2012 
n=59 



-49 — 



-34- 



■ 229- 



Second copy of the questionnaire 
n=263 



Figure 1 . Study population. 

visual analog scale (Lo et al. 2001). It has been cross-cultur- 
ally adapted and translated into Danish according to the rec- 
ommendation by Guillemin et al. (1993). The visual analog 
scale ranges from 0 to 100, with 100 as the worst score. The 
questions are divided into 4 domains (physical symptoms, 
sports and work, lifestyle, and emotions). The total score for 
all domains is summed to obtain a total score (0-100). The 
higher the score, the worse the outcome. 

Study population 

We included 787 patients reported to the DSR from Jan 1, 
2008 to Dec 31, 2008 (Figure 1). All had a Danish civil reg- 
istry number (CPR number) and an address in the Danish 
national population registry (CPR registry). We excluded 51 
operations where the patient had died within 12 months of sur- 
gery or had been reoperated in the same shoulder within the 
first 12 months. The latter exclusion criterion was due to DSR 
protocol, whereby patients reoperated within 1 year are sent 
just 1 questionnaire 12 months after the reoperation. Duplicate 
registrations in same procedure and obvious reporting errors 
were identified and corrected. 

Methods 

Age, sex, diagnosis, geographical region, and type of operation 
for all patients and WOOS score for responders and incom- 
plete responders were available in the DSR. Non-responders 
and incomplete responders were sent a second copy of the 
questionnaire in January 2012. The postal addresses were 
assessed through the CPR registry using the unique civil reg- 
istry number of each patient. Non-responders of the postal 
reminder were contacted by telephone after 3 weeks and 



Analysis and statistics 

For analysis, incomplete responders were 
grouped with responders. Responding and non- 
responding cohorts were compared with regard 
to the predefined variables age, sex, diagnosis, 
geographical region, and reoperation rate. Stu- 
dent's t-test was used for continuous data and 
chi-square test was used for nominal data using 
the continuity correction for 2 x 2 tables. In post 
hoc explorative analyses with multiple com- 
parisons, Bonferroni correction was applied. 
Responders to the first questionnaire, respond- 
ers to the postal reminder, and responders after 
telephone contact were compared with regard to 
total WOOS score and WOOS subscales using 
the Mann-Whitney U-test. A clinically relevant 
difference in outcome was defined as a differ- 
ence in total WOOS score of a 10 points based 
on the opinion of the surgeons at our depart- 
ment. Differences in outcome scores or pre- 
defined variables was considered statistically significant at 
p-values of < 0.05. All p-values were 2-sided. 95% confidence 
intervals (CIs) for median differences were calculated with the 
bootstrapping method using the R bootstrapping package. We 
used SPSS for Windows version 20.0 for all other statistical 
analyses. 

The study was approved by the Danish Data Protection 
Agency (2007-58-0015 / HEH.750.41-12). 



Results 

787 patients who had been sent a postal questionnaire 12 
months postoperatively were enrolled in the study. Mean age 
was 69 (24-96) years and 72% were women (Table 1). 94% 
were primary operations and 6% were revisions. 82% of the 
patients had hemiprostheses, 13% had reverse prostheses, 4% 
had total shoulder prostheses, and for 2% the type of prosthe- 
ses was not reported. 

Response rates 

465 patients (65%) responded to the original questionnaire 
12 months postoperatively (6% had filled it in incompletely). 
156 patients responded to the postal reminder, increasing 
the response rate for all patients included to 80% (with 3% 
incomplete) (Figure 2). Telephone follow-up resulted in 27 
responses out of 54 patients reached, further increasing the 
overall response rate to 82% (with 2% incomplete). 

Outcome scores 

No differences in total WOOS scores and WOOS subscales 
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Reminder with second copy of the questionnaire 
n=263 



Excluded (n=6): 

- failed mail delivery, 1 

- revision after the first 1 2 months post- 
operatively but before January 201 2, 5 



Returned reminders 
n=164 



Non-returned reminders 
n=93 



Blank questionnaires (n=8): 

- co-morbidities that impair 
cognitive function, 5 

- refused to participate, 3 



Completely filled 
in reminders 
n=142 



Incompletely filled 
in reminders 
n=14 



Incomplete responders 
without response to reminder 



Non-responders 
after postal reminder 
n=92 



X 



; i 

Telephone follow-up 
54 patients were reached 
, I 



Incomplete responders without response 
to postal and telephone reminder 
n=7 



Complete responses 
after telephone contact 
n=27 



Consistent 
non-responders 
n=67 



Figure 2. Responses after the reminder. 

between responders to the original questionnaire, responders 
to the postal reminder, and responders after telephone con- 
tact reached statistical significance (Table 1). The median 
total WOOS score was 7.5 (CI: -1.9 to 15) percentage points 
higher for responders to the postal reminder than for respond- 
ers to the first questionnaire (p = 0.3). For responders after 
telephone contact, the median total WOOS score was 8.6 (CI: 
-8.7 to 23) percentage points higher than for responders after 
the questionnaire or the postal reminder (p = 0.2). The CIs for 
differences in median total WOOS scores included the value 
0 and the minimal clinically relevant difference: 10. Thus, 



we could not determine whether there was 
a clinically relevant difference in outcome 
between responders and non-responders. 



Predefined variables 

There was no statistical evidence for dif- 
ferences in mean age, sex, or geographical 
region between responders and non-respond- 
ers to the original questionnaire (Table 2). 
Response rate was related to diagnosis (p 
= 0.002), but post hoc explorative analysis 
comparing the 4 major indications — arthri- 
tis, arthrosis, fracture, and cuff arthropathy, 
involving 93% of the patients — revealed no 
differences in response rate between these 
subgroups (uncorrected p-values: 0.2, 0.6, 
0.8, 0.8, 1.0, 0.9; all corrected p-values 1). 
After the postal reminder, no statistically 
significant difference was found between 
responders and non-responders with respect 
to sex, diagnosis, geographical region, and 
reoperation status. Non-responders to the 
postal reminder were younger, with a mean 
age of 65 (24-89) years as opposed to a 
mean age of 69 (26-94) years for respond- 
ers (p = 0.002). Consistent non-responders showed no statis- 
tically significant differences in sex, diagnosis, geographical 
region, or reoperation status compared to the pooled group of 
all responders. However, they were generally younger than 
responders, with a mean age of 64 (24-87) years as compared 
to 69 (26-94) years for responders (p = 0.001). 

Revision and outcome 

A post hoc analysis revealed a median total WOOS score of 
57 (IQR: 32-68) for revision patients and 41 (IQR: 17-63) for 
primarily operated patients. This difference of 16 (CI: 0-23) 



Table 1 . Functional outcome: WOOS scores. Values are median (interquartile range) 







A 




B 




C 




D 


E 


F 


G 


H 


WOOS Total 


40 


(17-63) 


47 


(21-65) 


49 


(22-68) 


41 


(18-64) 


7.5 (-1.9 to 15) 


0.3 


8.6 (-8.7 to 24) 


0.2 


WOOS Physical symptoms 


30 


(11-50) 


34 


(11-57) 


38 


(21-57) 


32 


(11-54) 


3.5 (-2.2 to 9.8) 


0.3 


10 (-8.2 to 24) 


0.2 


WOOS Sport and work 


42 


(19-63) 


54 


(23-78) 


72 


(29-91) 


53 


(23-77) 


4.4 (-4.0 to 15) 


0.3 


20 (-12 to 29) 


0.08 


WOOS Lifestyle 


44 


(19-71) 


51 


(23-72) 


57 


(26-80) 


46 


(20-72) 


6.4 (-5.3 to 15) 


0.4 


3.9 (-3.5 to 26) 


0.5 


WOOS Emotions 


28 


(5-66) 


37 


(7-68) 


47 


(6-69) 


32 


(5-67) 


9.7 (-5.0 to 22) 


0.2 


18 (-19 to 35) 


0.5 



A I. Responders to the first questionnaire a n = 509 

B II. Responders to the reminder questionnaire a n = 156 

C III. Responders after telephone follow-up n = 27 

D All Responders (original + reminder questionnaire + telephone) n = 656 

E Difference in medians (I vs. II) (95% CI) 

F P-value of difference (I vs. II) 

G Difference in medians (I + II vs. Ill) (95% CI) 

H P-value of difference (I + II vs. Ill) 

a Incomplete responses to the original questionnaire are part of group I, and if the patients responded to the postal reminder, their second 
answer is part of group II. 
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Table 2. Comparison of predefined variables 





A 


B 


c 


D 




p 


G H 


Anp mpan (95% Oh 


69 (Fj8—7C)\ 




68 (67-701 


KR (62-681 


64 (61-681 


0.09 


0 002 0 001 

\J . \J\J C- \J.\J\J 1 


Spx (%1 












0.2 


0.6 0.3 


F 


565 (721 


374 (74) 


191 (691 


64 (69) 


44 (66) 






M 


222 (28) 


1 35 (27) 


87 (31 ) 


29 (31 ) 


23 (34) 






niannn^i^ (%>\ 

1 ' 1 Gl LJ 1 IUU 1 0 l/OJ 












0.002 


0.3 0.5 


Arthritic; 

rAI LI II 1 Llo 


?1 (41 


20 (41 




5 (51 


3 (51 






Odt pna rthrnQic; 
vcsicual li 1 1 uoio 


231 (29) 


1 48 (29) 


art noi 


?4 (?61 


17 (?51 






Frarti irp 

1 1 CHjLU 1 C 


406 (521 


275 (54) 


131 (47) 


49 (53) 


?9 (5R1 






Arthrnnathv of thp rotator ruff 

/Y 1 LIIIULJClLliy ul LI 1 C 1 ULuL^I LiUII 


64 (81 


41 (81 


23 (81 


5 (51 


o ^vjy 






Necrosis of the humeral head 


21 (3) 


15(3) 


6(2) 


2(2) 


1 (2) 






Other diagnosis 


25 (3) 


7(1) 


18(7) 


6(7) 


4(6) 






Unknown 


9(1) 


3(1) 


6(2) 


2(2) 


0(0) 






Geographical region (%) 












0.4 


0.2 0.2 


The capital area (Copenhagen) 


223 (28) 


140 (28) 


83 (16) 


29 (31) 


23 (34) 






North of Jutland 


82 (10) 


56 (11) 


26 (9) 


6(7) 


5(8) 






The middle part of Jutland 


265 (34) 


165 (32) 


100 (36) 


37 (40) 


27 (40) 






South of Jutland and Fyn 


143 (18) 


102 (20) 


41 (15) 


11 (12) 


6(9) 






Sealand 


74 (9) 


46 (9) 


28 (10) 


10(11) 


6(9) 






Reoperated (%) 












0.000 


1.00 1.00 


Yes 


50 (6) 


11 (2) 


39 (14) 


6(7) 


4(6) 






No 


737 (94) 


498 (98) 


239 (86) 


87 (94) 


63 (94) 







A All (n = 787) 

B I. Responders (n = 509) 

C II. Non-responders (n = 278) 

D III. Non-responders after reminder (n = 92), a subgroup of non-responders (II) 

E IV. Consistent non-responders (n = 67), a subgroup of III. 

F P-value of difference (I vs. II) 

G P-value of difference, group III compared to responders of the original and reminder questionnaire (initial responders + responders to the 
reminder). 

H P-value of difference, group IV compared to all responders (initially + after reminder + after telephone contact). 



percentage points was clinically and statistically significant 
(p = 0.03). When revision patients were left out of the analy- 
ses, the median total WOOS scores were only 2 (CI: -8~> to 12) 
percentage points higher for responders to the postal reminder 
than for responders to the original questionnaire (p = 0.2). On 
the other hand, the median total WOOS scores for responders 
after telephone contact were 10 (CI: 5-25) percentage points 
higher than the median total WOOS scores for initial respond- 
ers and responders to the postal reminder (p = 0.1). 



Discussion 

We did not find any statistical evidence for different outcome 
scores for respondents and non-respondents at one-year fol- 
low-up in patients after shoulder replacement. However, a 
trend of worse outcome scores for non-responders was noted. 
This trend is partly explained by the fact that there was a 
higher proportion of revision patients in the group of non- 
responders; revision was associated with inferior outcome 
scores. When revision patients were excluded, the trend of 
worse outcome in responders to the postal reminder disap- 
peared. However, for the subgroup of non-responders that 
responded after telephone contact, the trend of worse outcome 
scores persisted. This trend might have become statistically 



significant if the group of responders after telephone contact 
had been larger. We were not able to reject or confirm that the 
trend of worse outcome scores was clinically relevant accord- 
ing to our predefined minimal clinically relevant difference. 
Younger patients were less likely to respond. 

Our results do not agree with the findings in clinical stud- 
ies, in which primary non-responders to PROMs have reported 
worse outcome (Norquist et al. 2000, Kim et al. 2004, Kwon 
et al. 2010). Rather, our findings are in line with those of Sol- 
berg et al. (201 1), who found similar patient-reported outcome 
for responders and non-responders in the Norwegian spine 
surgery registry. They assessed sequential patient-reported 
outcomes 3 times over a 2-year follow-up period, and defined 
non-responders as those failing to respond after 1 reminder 
had been sent. In a younger population with a mean age of 42 
years, they found a higher proportion of non-responders than 
we did. They found that non-responders were younger and had 
fewer complications. 

One reason for the discrepancy in findings between clinical 
studies and large registry studies could be that the study design, 
the setting, and patient selection differ. In clinical studies, rig- 
orous efforts can be made to trace the non-responders, and eli- 
gibility criteria can restrict the population studied to avoid the 
influence of factors that might affect patient-reported outcome 
and response behavior. 
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One strength of the present study was the availability of 
the unique civil registry number, which allows linkage of the 
individuals registered in the registry database to the national 
population registry. As the national population registry is con- 
tinuously updated regarding postal address and date of death, 
it enabled us to reach almost all the living non-responders by 
postal questionnaire. It has been documented that the use of 
both telephone and postal questionnaires can introduce infor- 
mation bias (Norquist et al. 2000, Grimes and Schulz 2002, 
Ludemann et al. 2003). Thus, we did not interview the patients 
by telephone but only encouraged them to respond by mail. 

This study had some limitations. Firstly, we were unable to 
reach all non-responders to the postal reminder by telephone, 
because there is no complete database with telephone num- 
bers. Secondly, the time delay between the first questionnaire 
12 months postoperatively and the postal reminder in January 
2012 limited the completeness of follow-up, as more patients 
were lost to follow-up due to death or reoperation. Thirdly, 
we cannot exclude improvement or deterioration in WOOS 
scores in the period between the first questionnaire and the 
reminder. A smaller, gradual deterioration in joint evalua- 
tion scores has been reported after hip and knee replacement 
(Ritter et al. 2004). Due to differences in weight load on joints 
of the upper and lower extremities, these results may not be 
directly extrapolated to our cohort, and outcome of shoulder 
replacement is considered to be almost stable after one year 
(Wirth et al. 2006, Ohl et al. 2010, Cazeneuve and Cristofari 
2011). Thus, we presume that deterioration or improvement 
in WOOS scores as a result of the time delay was of minor 
importance in our study. Hence, if a clinically relevant differ- 
ence in patient-reported functional outcome existed between 
responders and non-responders 12 months postoperatively, we 
would expect the difference to persist after 3 years. 

As patients who consistently failed to respond were gen- 
erally younger, it is likely that communication by e-mail or 
social media might have reached this subgroup of patients. 
Moreover, our data suggest that non-responders are a het- 
erogeneous group, and revision proved to be an important 
factor related to both response behavior and outcome. Sev- 
eral other demographic factors, socioeconomic factors, and 
health-related factors (such as co-morbidity) might influence 
response behavior and patient-reported outcome. These fac- 
tors should be investigated in future studies of patients who do 
not respond to PROMs. 

In conclusion, non-responders did not appear to bias the 
overall results after shoulder replacement despite there being 
a trend of worse outcome for a subgroup of non-responders. 
Age and revision rate were important factors related to non- 
response, and these should be considered when PROMs in 
registries are being interpreted. We recommend the use of 
reminders in arthroplasty registries using PROMs, as remind- 
ers raised the response rates markedly. 
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